Vehicle drive assist apparatus

ABSTRACT

A vehicle drive assist apparatus to be applied to a vehicle executes at least emergency braking control for avoiding collision of the vehicle with a recognized object. The vehicle drive assist apparatus includes a surrounding environment recognition device and a traveling control unit. The surrounding environment recognition device includes a recognizer that recognizes a surrounding environment around the vehicle, and a feature information acquirer that acquires feature information of a target object in the recognized surrounding environment. The traveling control unit centrally controls the entire vehicle. The traveling control unit includes a determiner that determines, based on the acquired feature information, whether the target object has a possibility of hindering traveling of the vehicle. The traveling control unit continues normal traveling control for the vehicle when the target object is determined to have no possibility of hindering the traveling of the vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from Japanese Patent ApplicationNo. 2022-008109 filed on Jan. 21, 2022, the entire contents of which arehereby incorporated by reference.

BACKGROUND

The disclosure relates to a vehicle drive assist apparatus that performsdrive assist control for avoiding collision with an obstacle or the likebased on surrounding environment information acquired by using anon-board camera device and the like.

In the field of vehicles such as automobiles, autonomous driving controltechnologies have been developed to cause the vehicles to autonomouslytravel without driving operations of drivers who drive the vehicles.Various drive assist apparatuses using the autonomous driving controltechnologies have been proposed and put into practical use to performvarious types of traveling control for assisting driving operations ofdrivers.

Related-art drive assist apparatuses use sensing devices such as anon-board camera device and an on-board radar device as a surroundingenvironment recognition device that recognizes the surroundingenvironment around a vehicle and acquires the surrounding environment assurrounding information. The on-board camera device recognizes thesurrounding environment around the vehicle based on acquired image data.The on-board radar device recognizes the surrounding environment aroundthe vehicle by outputting radio waves to the surroundings of thevehicle, receiving the reflected radio waves from objects, and analyzingthe received radio waves.

Each related-art drive assist apparatus causes the vehicle to travelwhile recognizing the surrounding environment around the vehicle byusing the sensing devices. For example, when an obstacle that may hinderthe traveling of the vehicle is recognized on a traveling path of thevehicle, emergency braking control or emergency steering control isperformed to prevent collision between the vehicle and the obstacle.Thus, the vehicle can continue to travel safely. The control technologysuch as a drive assist apparatus having an obstacle avoidance controlfunction for avoiding collision with an obstacle has variously beenproposed and put into practical use as in, for example, JapaneseUnexamined Patent Application Publication No. 2019-18733.

SUMMARY

An aspect of the disclosure provides a vehicle drive assist apparatus tobe applied to a vehicle. The vehicle drive assist apparatus isconfigured to execute at least emergency braking control for avoidingcollision of the vehicle with a recognized object. The vehicle driveassist apparatus includes a surrounding environment recognition deviceand a traveling control unit. The surrounding environment recognitiondevice includes a recognizer and a feature information acquirer. Therecognizer is configured to recognize a surrounding environment aroundthe vehicle. The feature information acquirer is configured to acquirefeature information of a target object in the recognized surroundingenvironment. The traveling control unit is configured to centrallycontrol a whole of the vehicle. The traveling control unit includes adeterminer configured to determine, based on the acquired featureinformation, whether the target object has a possibility of hinderingtraveling of the vehicle. The traveling control unit is configured tocontinue normal traveling control for the vehicle when target objectdoes is determined to have no possibility of hindering the traveling ofthe vehicle.

An aspect of the disclosure provides a vehicle drive assist apparatus tobe applied to a vehicle. The vehicle drive assist apparatus isconfigured to execute at least emergency braking control for avoidingcollision of the vehicle with a recognized object. The vehicle driveassist apparatus includes a surrounding environment recognition deviceand circuitry. The surrounding environment recognition device has asensor and is configured to recognize a surrounding environment aroundthe vehicle. The detector is configured to acquire feature informationof a target object in the recognized surrounding environment. Thecircuitry is configured to centrally control a whole of the vehicle. Thecircuitry is configured to, based on the acquired feature information,determine whether the target object has a possibility of hinderingtraveling of the vehicle. The circuitry is configured to, upondetermining that the target object is has no possibility of hinderingthe traveling of the vehicle, continue normal traveling control for thevehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of the disclosure and are incorporated in and constitute apart of this specification. The drawings illustrate example embodimentsand, together with the specification, serve to describe the principlesof the disclosure.

FIG. 1 is a block diagram illustrating a schematic configuration of adrive assist apparatus according to an embodiment of the disclosure;

FIG. 2 is a conceptual diagram of how a vehicle including the driveassist apparatus according to the embodiment of the disclosurerecognizes obstacles etc. ahead;

FIG. 3 is a diagram illustrating a display example of a forward imageacquired by an on-board camera device of the vehicle including the driveassist apparatus according to the embodiment of the disclosure;

FIG. 4 is a flowchart illustrating a flow of operations of the driveassist apparatus according to the embodiment of the disclosure; and

FIG. 5 is a flowchart illustrating a flow of operations of a driveassist apparatus according to an embodiment of the disclosure.

DETAILED DESCRIPTION

Examples of the objects to be recognized by the related-art drive assistapparatus include various road marks such as lane lines and planarstructures such as manholes (hereinafter referred to simply as “lanelines etc.”). There are three-dimensional objects on roads, such aspermanently installed stationary objects typified by roadsidecurbstones, guardrails, traffic signs, utility poles, and billboards,and moving objects typified by pedestrians, bicycles, other vehicles,and animals.

Further, there are various three-dimensional objects on roads, such astemporarily installed stationary objects typified by signage and trafficcones near construction sites, and objects that have fallen from luggagespaces of other vehicles (hereinafter referred to collectively as“obstacles etc.”).

For example, empty plastic bags may float in the air due to wind or thelike because of their small mass (hereinafter referred to as “airborneobjects”). The surrounding environment recognition device of the driveassist apparatus may recognize the various objects described above.

When a three-dimensional object that satisfies a predetermined conditionamong the various objects described above is recognized on the travelingpath of the vehicle, the related-art drive assist apparatus recognizesthe three-dimensional object as an object that may hinder the travelingof the vehicle. In this case, obstacle avoidance control includingemergency braking control or emergency steering control is executed toavoid collision with the recognized obstacle.

The related-art drive assist apparatus may execute predeterminedobstacle avoidance control even if the recognized object is, forexample, the airborne object.

The airborne object is unlikely to hinder the traveling of the vehicle.When the airborne object is recognized and an obstacle avoidance actionis executed, the driver of the vehicle may have discomfort.

It is desirable to provide a vehicle drive assist apparatus configuredto perform drive assist control for avoiding collision with obstaclesetc. based on surrounding environment information acquired by using anon-board camera device and the like, and configured not to immediatelyexecute obstacle avoidance control including emergency braking controlwhen an airborne object or the like is recognized during traveling of avehicle, thereby constantly executing smooth traveling control withoutcausing driver's discomfort.

Embodiments of the disclosure are described below. The drawings for usein the following description are schematic drawings. To illustrateconstituent elements in recognizable sizes in the drawings, dimensionalrelationships and scales of members may be varied among the constituentelements. The embodiments of the disclosure are not limited to theembodiments in the drawings in terms of the numbers, shapes, sizeratios, and relative positional relationships of the constituentelements in the drawings. Throughout the present specification and thedrawings, elements having substantially the same function andconfiguration are denoted with the same numerals to avoid any redundantdescription.

The schematic configuration of a drive assist apparatus according to afirst embodiment of the disclosure is described with reference to FIG. 1. FIG. 1 is a block diagram illustrating the schematic configuration ofthe drive assist apparatus according to the first embodiment of thedisclosure. As illustrated in FIG. 1 , a drive assist apparatus 1 ofthis embodiment basically has a configuration substantially similar tothose of related-art drive assist apparatuses of the same type.Therefore, the following description is overall description of the driveassist apparatus 1 of this embodiment.

The drive assist apparatus 1 of this embodiment includes a camera unit10 fixed to an upper central part of a front area in a cabin of avehicle including the drive assist apparatus 1 (hereinafter referred tosimply as “vehicle”). In one embodiment, the camera unit 10 may serve asan on-board camera device.

The camera unit 10 includes a stereo camera 11, an image processing unit(IPU) 12, an image recognition unit (image recognition ECU) 13, and atraveling control unit (traveling ECU) 14.

In one embodiment, the stereo camera 11 is a device that may serve as arecognizer configured to recognize a surrounding environment around avehicle. The stereo camera 11 includes a main camera 11 a and asubcamera 11 b. For example, the main camera 11 a and the subcamera 11 bare disposed in the cabin of the vehicle at bilaterally symmetricalpositions across the center in a vehicle width direction to face aforward side (in the traveling direction). For example, the main camera11 a and the subcamera 11 b each include a CMOS image sensor, andgenerate a stereoscopic image by acquiring two images of a surroundingenvironment in an external forward area within a predetermined rangefrom different viewpoints in every predetermined synchronous imagingperiod.

The IPU 12 performs predetermined image processing for surroundingenvironment image data obtained by the stereo camera 11 (image datashowing the surrounding environment during traveling of the vehicle) todetect edges of various target objects such as objects in the image andlane lines on a road surface. Thus, the IPU 12 recognizes thethree-dimensional objects and the lane lines around the vehicle. The IPU12 acquires distance information from positional deviation amounts ofcorresponding edges in the right and left images, and generates imageinformation including the distance information (distance imageinformation).

The image recognition ECU 13 obtains, based on the distance imageinformation received from the IPU 12, a road curvature [1/m] betweenright and left lane lines of a road where the vehicle is traveling(vehicle traveling road) and a width between the right and left lanelines (lane width). Various methods are known to obtain the roadcurvature and the lane width. For example, the image recognition ECU 13obtains the road curvature in such a manner that right and left lanelines are recognized by binarization using a difference in brightnesslevels based on surrounding environment information and the curvaturesof the right and left lane lines are obtained for each predeterminedsection by using a curve approximation expression based on theleast-square method. The image recognition ECU 13 calculates the lanewidth from a difference in the curvatures of the right and left lanelines.

The image recognition ECU 13 calculates, based on the curvatures of theright and left lane lines and the lane width, a vehicle lateral positiondeviation that is a distance from a lane center to the center of thevehicle in the vehicle width direction.

The image recognition ECU 13 performs predetermined pattern matching forthe distance image information to recognize three-dimensional objectssuch as guardrails along the road, curbstones, and surrounding vehicles.In the recognition of three-dimensional objects, the image recognitionECU 13 recognizes, for example, types of the three-dimensional objects,heights of the three-dimensional objects, distances from thethree-dimensional objects, speeds of the three-dimensional objects,relative speeds between the three-dimensional objects and the vehicle,and relative distances between the three-dimensional objects (forexample, a lateral distance between a roadside curbstone and a lane linenearby).

Various types of information recognized by the image recognition ECU 13are output to the traveling ECU 14 as first surrounding environmentinformation.

In one embodiment, the image recognition ECU 13 of the drive assistapparatus 1 may serve as a surrounding environment recognition deviceconfigured to recognize a first surrounding environment around thevehicle in cooperation with the stereo camera 11 and the IPU 12.

The traveling ECU 14 is a control unit that centrally controls the driveassist apparatus 1. Various control units such as a cockpit control unit(CP_ECU) 21, an engine control unit (E/G_ECU) 22, a transmission controlunit (T/M_ECU) 23, a brake control unit (BK_ECU) 24, and a powersteering control unit (PS_ECU) 25 are coupled to the traveling ECU 14via an internal communication network such as a controller area network(CAN).

Various sensors such as a locator unit 36, on-board radar devices 37(right front side sensor 37 rf, left front side sensor 371 f, right rearside sensor 37 rr, and left rear side sensor 371 r), and a rear sensor38 are coupled to the traveling ECU 14. A human-machine interface (HMI)31 disposed near a driver's seat is coupled to the CP_ECU 21. Forexample, the HMI 31 includes a switch for giving instructions to executevarious types of drive assist control, a mode selection switch forswitching driving modes, a steering touch sensor that detects a steeringwheel holding state of a driver, a driver monitoring system (DMS) thatperforms facial authentication of the driver and detects a line ofsight, a touch panel display, a combination of meters, and aloudspeaker. In one embodiment, the touch panel display may serve as adisplay panel.

In response to a control signal from the traveling ECU 14, the CP_ECU 21notifies the driver as appropriate by display and sound through the HMI31 about various types of information related to, for example, variousalerts for a preceding vehicle, the status of the drive assist control,and the surrounding environment around the vehicle. The CP_ECU 21outputs, to the traveling ECU 14, various types of information input bythe driver through the HMI 31, such as ON/OFF operations on varioustypes of drive assist control.

For example, a throttle actuator 32 of an electronically controlledthrottle is coupled to an output side of the E/G_ECU 22. Various sensorssuch as an accelerator sensor (not illustrated) are coupled to an inputside of the E/G_ECU 22.

The E/G_ECU 22 controls drive of the throttle actuator 32 based on, forexample, a control signal from the traveling ECU 14 or detection signalsfrom various sensors. Thus, the E/G_ECU 22 adjusts the intake amount ofan engine to generate desired engine power. The E/G_ECU 22 outputs, tothe traveling ECU 14, signals of an accelerator operation amount and thelike detected by various sensors.

A hydraulic control circuit 33 is coupled to an output side of theT/M_ECU 23. Various sensors such as a shift position sensor (notillustrated) are coupled to an input side of the T/M_ECU 23. The T/M_ECU23 performs hydraulic control for the hydraulic control circuit 33 basedon, for example, a signal of an engine torque estimated by the E/G_ECU22 and detection signals from various sensors. Thus, the T/M_ECU 23changes the engine power at a desired speed ratio by operating, forexample, friction engagement elements and pulleys in an automatictransmission. The T/M_ECU 23 outputs, to the traveling ECU 14, signalsof a shift position and the like detected by various sensors.

A brake actuator 34 for adjusting brake fluid pressures to be output tobrake wheel cylinders in individual wheels is coupled to an output sideof the BK_ECU 24. Various sensors such as a brake pedal sensor, a yawrate sensor, a longitudinal acceleration sensor, and a vehicle speedsensor (not illustrated) are coupled to an input side of the BK_ECU 24.

The BK_ECU 24 controls drive of the brake actuator 34 based on a controlsignal from the traveling ECU 14 or detection signals from varioussensors. Thus, the BK_ECU 24 generates, for the wheels as appropriate,braking forces for forcible braking control and yaw rate control on thevehicle. The BK_ECU 24 outputs, to the traveling ECU 14, signals of abrake operation status, a yaw rate, a longitudinal acceleration, avehicle speed, and the like detected by various sensors.

An electric power steering motor 35 for applying a steering torque of arotational force from a motor to a steering mechanism is coupled to anoutput side of the PS_ECU 25. Various sensors such as a steering torquesensor and a steering angle sensor are coupled to an input side of thePS_ECU 25.

The PS_ECU 25 controls drive of the electric power steering motor 35based on a control signal from the traveling ECU 14 or detection signalsfrom various sensors. Thus, the PS_ECU 25 generates the steering torquefor the steering mechanism. The PS_ECU 25 outputs, to the traveling ECU14, signals of a steering torque, a steering angle, and the likedetected by various sensors.

The locator unit 36 includes a GNSS sensor 36 a and a high-accuracy roadmap database (road map DB) 36 b.

The GNSS sensor 36 a measures the position (latitude, longitude, andaltitude) of the vehicle by receiving positioning signals frompositioning satellites.

The road map DB 36 b is a large-capacity storage medium such as an HDDor an SSD, and stores high-accuracy road map information (dynamic map).For example, the road map DB 36 b stores lane width data, lane centerposition coordinate data, lane azimuth angle data, and speed limits aslane data for use in autonomous driving. The stored lane data includespieces of data for several-meter intervals in each lane on the road map.The road map DB stores information on various facilities and parkinglots. Based on, for example, a request signal from the traveling ECU 14,the road map DB 36 b outputs road map information in a set range aroundthe vehicle position measured by the GNSS sensor 36 a to the travelingECU 14 as third surrounding environment information.

In one embodiment, the road map DB 36 b of the drive assist apparatus 1may serve as the surrounding environment recognition device configuredto recognize a third surrounding environment around the vehicle incooperation with the GNSS sensor 36 a.

The right front side sensor 37 rf, the left front side sensor 371 f, theright rear side sensor 37 rr, and the left rear side sensor 371 rconstitute the on-board radar devices 37. For example, the sensors aremillimeter wave radars.

Each millimeter wave radar detects a three-dimensional object such as apedestrian or a vehicle traveling side by side and a structure(three-dimensional object such as a curbstone, a guardrail, a wall of abuilding, or a plant) along a roadside (for example, an end at a roadshoulder) by outputting radio waves and analyzing the reflected radiowaves from the objects. Each millimeter wave radar also detects athree-dimensional obstacle on a road. For example, each radar detects awidth of the three-dimensional object, a position of a representativepoint of the three-dimensional object (position and distance relative tothe vehicle), and a relative speed as specific information related tothe three-dimensional object.

For example, the right front side sensor 37 rf and the left front sidesensor 371 f are disposed on right and left sides of a front bumper. Theright front side sensor 37 rf and the left front side sensor 371 fdetect, as second surrounding environment information, three-dimensionalobjects in right and left obliquely forward and side areas around thevehicle. Those areas are difficult to recognize from an image capturedby the stereo camera 11.

For example, the right rear side sensor 37 rr and the left rear sidesensor 371 r are disposed on right and left sides of a rear bumper. Theright rear side sensor 37 rr and the left rear side sensor 371 r detect,as the second surrounding environment information, three-dimensionalobjects in right and left obliquely rearward and side areas around thevehicle. Those areas are difficult to recognize by the right front sidesensor 37 rf and the left front side sensor 371 f.

In one embodiment, the on-board radar devices 37 (right front sidesensor 37 rf, left front side sensor 371 f, right rear side sensor 37rr, and left rear side sensor 371 r) of the drive assist apparatus 1 mayserve as the surrounding environment recognition device configured torecognize a second surrounding environment around the vehicle. Thepieces of information acquired by the sensors 37 rf, 371 f, 37 rr, and371 r are sent to the image recognition ECU 13.

Examples of the rear sensor 38 include a sonar device. For example, therear sensor 38 is disposed on the rear bumper. The rear sensor 38detects three-dimensional objects in a rearward area behind the vehicleas fourth surrounding environment information. This area is difficult torecognize by the right rear side sensor 37 rr and the left rear sidesensor 371 r.

In one embodiment, the rear sensor 38 of the drive assist apparatus 1may serve as the surrounding environment recognition device configuredto recognize a fourth surrounding environment around the vehicle.

Coordinates of the external targets in the first surrounding environmentinformation recognized by the camera unit 10 including the imagerecognition ECU 13, the third surrounding environment informationrecognized by the locator unit 36, the second surrounding environmentinformation recognized by the on-board radar devices 37 (right frontside sensor 37 rf, left front side sensor 371 f, right rear side sensor37 rr, and left rear side sensor 371 r), and the fourth surroundingenvironment information recognized by the rear sensor 38 are convertedby the traveling ECU 14 into coordinates in a three-dimensionalcoordinate system having its origin at the center of the vehicle.

The traveling ECU 14 has driving modes such as a manual driving mode, afirst traveling control mode, a second traveling control mode, and alimp home mode. The traveling ECU 14 can selectively switch the drivingmodes based on, for example, a status of operation on the mode selectionswitch of the HMI 31.

The manual driving mode is a driving mode in which the driver is assumedto hold the steering wheel. In this driving mode, the vehicle travels bydriving operations of the driver, such as a steering operation, anaccelerator operation, and a brake operation.

The first traveling control mode is also a driving mode in which thedriver is assumed to hold the steering wheel. That is, the firsttraveling control mode is a so-called semi-autonomous driving mode ordrive assist mode in which the vehicle travels along a target travelingroute by combining, for example, adaptive cruise control (ACC), activelane keep centering (ALKC) control, and active lane keep bouncingcontrol as appropriate through control of, for example, the E/G_ECU 22,the BK_ECU 24, and the PS_ECU 25 while reflecting the driving operationsof the driver.

The adaptive cruise control (ACC) is basically performed based on thefirst surrounding environment information input from the imagerecognition ECU 13. For example, the adaptive cruise control (ACC) isperformed based on preceding vehicle information in the firstsurrounding environment information from the image recognition ECU 13.

The active lane keep centering control and the active lane keep bouncingcontrol are basically performed based on the first surroundingenvironment information and the third surrounding environmentinformation input from one or more of the image recognition ECU 13 andthe locator unit 36. For example, the active lane keep centering controland the active lane keep bouncing control are performed based on laneline information in the third surrounding environment information fromthe image recognition ECU 13 or the locator unit 36.

The second traveling control mode is an autonomous driving mode thatrealizes a so-called hands-off function in which the vehicle travelsalong a target route (route map information) by combining, for example,the adaptive cruise control, the active lane keep centering control, andthe active lane keep bouncing control as appropriate through control of,for example, the E/G_ECU 22, the BK_ECU 24, and the PS_ECU 25 withoutthe steering wheel holding by the driver, the accelerator operation, andthe brake operation.

In the limp home mode, the vehicle is automatically stopped, forexample, at a side strip when the vehicle traveling in the secondtraveling control mode cannot continue the traveling in this mode andthe driver cannot take over the driving operation (that is, the modecannot be switched to the manual driving mode or the first travelingcontrol mode).

In each of the driving modes described above, the traveling ECU 14determines whether to execute obstacle avoidance control involvingautonomous emergency braking (AEB: collision damage reduction braking)control and emergency steering control in response to recognition of athree-dimensional obstacle such as a preceding vehicle or a fallenobject on a vehicle traveling road with a strong possibility ofcolliding with the vehicle, and executes predetermined control asappropriate.

All or part of the locator unit 36, the image recognition ECU 13, thetraveling ECU 14, the CP_ECU 21, the E/G_ECU 22, the T/M_ECU 23, theBK_ECU 24, and the PS_ECU 25 are/is a processor including hardware.

For example, the processor is constituted by known components and theirperipheral devices including a central processing unit (CPU), a randomaccess memory (RAM), a read only memory (ROM), a non-volatile memory, anon-volatile storage, and a non-transitory computer readable medium.

The ROM, the non-volatile memory, and the non-volatile storage prestoresoftware programs to be executed by the CPU and fixed data such as datatables. The CPU reads the software programs stored in the ROM and thelike and executes the software programs by loading the software programsin the RAM. The software programs implement the functions of thecomponents and units (13, 14, 21 to 25, 36) by referring to varioustypes of data as appropriate.

The processor may be implemented by a semiconductor chip such as a fieldprogrammable gate array (FPGA). The components and units (13, 14, 21 to25, 36) may be implemented by electronic circuits.

The software programs may entirely or partly be recorded as computerprogram products in a non-transitory computer readable medium such as aportable sheet medium typified by a flexible disk, a CD-ROM, or aDVD-ROM, a card memory, a hard disk drive (HDD), or a solid state drive(SSD).

In one embodiment, a monocular camera may serve as the surroundingenvironment recognition device in place of (or in addition to) thestereo camera 11 in the camera unit 10. In one embodiment, a lightdetection and ranging (LiDAR) sensor may serve as the surroundingenvironment recognition device in place of (or in addition to) theon-board radar devices 37.

Operations of the drive assist apparatus 1 of this embodiment aredescribed with reference to FIG. 2 to FIG. 4 .

The drive assist apparatus 1 of this embodiment has a function ofassisting driving operations of the driver by executing, for example,the adaptive cruise control (ACC), the active lane keep centering (ALKC)control, and the active lane keep bouncing (ALKB) control.

The drive assist apparatus 1 of this embodiment also has a function ofassisting the driving by executing the obstacle avoidance control inresponse to recognition of an obstacle on a traveling road. In thiscase, the drive assist apparatus 1 of this embodiment determines whetherthe recognized obstacle has no possibility of hindering the traveling ofthe vehicle. When determination is made that the recognized obstacle hasno possibility of hindering the traveling of the vehicle, the obstacleavoidance control such as emergency braking is suppressed.

Examples of the objects to be recognized by the drive assist apparatus 1include various road marks such as lane lines and planar structures suchas manholes (hereinafter referred to simply as “lane lines etc.”). Theplanar structures, that is, the lane lines etc. are regarded as havingno possibility of hindering the traveling of the vehicle in therelated-art drive assist apparatuses.

There are three-dimensional objects on roads, such as permanentlyinstalled stationary objects typified by roadside curbstones,guardrails, traffic signs, utility poles, and billboards, and movingobjects typified by pedestrians, bicycles, other vehicles, and animals.Further, there are various three-dimensional objects on roads, such astemporarily installed stationary objects typified by signage and trafficcones near construction sites, and objects that have fallen from luggagespaces of other vehicles. Those three-dimensional objects are recognizedas objects that may hinder the traveling of the vehicle. Thosethree-dimensional objects are hereinafter referred to collectively as“obstacles etc.”

For example, empty plastic bags may float in the air due to wind or thelike because of their small mass (hereinafter referred to as “airborneobjects”). The airborne objects are three-dimensional objects but arerecognized as objects having no possibility of hindering the travelingof the vehicle based on their feature information.

Examples of the feature information of the airborne object include suchfeatures that the object is present in the air, that is, away from theroad surface, the appearance size (area in a recognition image) is verysmall (minimal), the weight is small and therefore the falling speed isvery low (the object falls at a low speed), the object falls not only inthe gravity direction (the direction varies upward, downward, rightward,or leftward due to wind or the like and therefore the falling speed mayalso vary), the appearance size or shape changes while the object isfalling (the shape changes due to wind or the like), and thetransparency is high (some plastic bags are colorless and transparent).

The feature information of the airborne object can be acquired, forexample, from a result of the predetermined image processing performedbased on an image acquired by the camera unit 10. In one embodiment, thecamera unit 10 that may serve as the surrounding environment recognitiondevice may also serve as a feature information acquirer configured toacquire feature information of a target object in the recognizedsurrounding environment. The feature information acquired by the featureinformation acquirer is sent to the traveling ECU 14. In one embodiment,the traveling ECU 14 may serve as a determiner configured to determinewhether the target object may hinder the traveling of the vehicle basedon the received feature information.

FIG. 2 and FIG. 3 are conceptual diagrams illustrating a situation inwhich the vehicle including the drive assist apparatus of thisembodiment recognizes an object ahead in the traveling direction whiletraveling on a road.

In FIG. 2 , reference symbol 100 represents a road where the vehicle istraveling. In FIG. 2 , reference symbol M represents the vehicleincluding the drive assist apparatus 1. The vehicle M includes thecamera unit 10 that may serve as the surrounding environment recognitiondevice and the on-board camera device constituting the drive assistapparatus 1. The camera unit 10 is fixed at a predetermined position inthe cabin of the vehicle M. In FIG. 2 , reference symbol V representsthe field of view (imaging range) of the stereo camera 11 of the cameraunit 10. The field of view V corresponds to an angle of view of an imageto be acquired by the camera unit 10. FIG. 2 illustrates the field ofview V in a height direction.

The vehicle M including the drive assist apparatus 1 is traveling alongthe road 100. The drive assist apparatus 1 continuously acquires imagesshowing the surrounding environment (forward area in the travelingdirection) at predetermined time intervals by using the camera unit 10,and recognizes the surrounding environment by performing thepredetermined image processing based on the acquired image data.

An airborne object 101 is present away from the surface of the road 100(at a height H from the road surface) within a range of the field ofview V ahead of the vehicle M in the traveling direction. The driveassist apparatus 1 of the vehicle M recognizes the airborne object 101.FIG. 2 conceptually illustrates this situation.

FIG. 3 illustrates a display example of an image acquired by the cameraunit 10 of the drive assist apparatus 1 of the vehicle M in thesituation of FIG. 2 .

In FIG. 3 , reference symbol 10A represents the image acquired by thecamera unit 10. The surroundings of the vehicle M (forward area in thetraveling direction) are acquired as an image within the range of theimage 10A. In FIG. 3 , reference symbol 100 represents the road wherethe vehicle M is traveling as in FIG. 2 . Reference symbol 101represents the airborne object as in FIG. 2 . Reference symbol 101 arepresents the shadow of the airborne object 101. Reference symbol 102represents a manhole installed on the road 100. Reference symbol 103represents a mark (marking line) on the road 100. Reference symbol 104represents a curbstone along the left edge of the road 100. Referencesymbol 105 represents a plant or a sidewalk along the left edge of theroad 100. Reference symbol 106 represents a boundary wall along theright edge of the road 100. Reference symbol 107 represents a utilitypole installed at the right edge of the road 100. Reference symbol 108represents the shadow of a utility pole or a cable (not illustrated).Reference symbol 109 represents a bicycle traveling toward the vehicle Mon the road 100. Reference symbol 110 represents traffic cones placedalong the side edges of the road 100.

In FIG. 2 and FIG. 3 , arrows X, Y, and Z indicate coordinate axes of aspace including the vehicle M. The arrow X indicates a coordinate axisalong the width direction of the vehicle M (lateral direction). In aforward view from the vehicle M, a leftward direction of the arrow X isa positive (+) direction. The arrow Y indicates a coordinate axis alongthe height direction of the vehicle M (vertical direction). In theforward view from the vehicle M, an upward direction of the arrow Y is apositive (+) direction. The arrow Z indicates a coordinate axis alongthe traveling direction of the vehicle M. A direction of the arrow Zfrom the forward side to the vehicle M is a positive (+) direction.

The drive assist apparatus 1 recognizes various objects in thesurrounding environment around the vehicle M based on image data showingthe image 10A.

Operations of the drive assist apparatus 1 of the vehicle M in thissituation are described with reference to a flowchart of FIG. 4 . Thefollowing processing flow shows characteristic processes unique to theembodiment of the disclosure among various processes in the drive assistcontrol to be executed by the drive assist apparatus 1 of thisembodiment.

The vehicle M is traveling along the road 100 in an ON state of thedrive assist control of the drive assist apparatus 1. The drive assistapparatus 1 continuously recognizes the surrounding environment by usingthe surrounding environment recognition device (for example, the cameraunit 10).

In Step S11 of FIG. 4 , the drive assist apparatus 1 checks whethervarious recognized objects include an object floating in the air(airborne object 101). Whether the recognized object is the airborneobject 101 is determined as follows.

For example, as illustrated in FIG. 2 , the relative positionalrelationship between the height position of the sensor (camera unit 10)of the vehicle M and the height position of the target object (airborneobject 101) is estimated based on image data. Thus, the height H of thetarget object (airborne object 101) from the road surface can becalculated. When the height H is equal to or larger than a predeterminedthreshold, the target object can be estimated as the airborne object101.

For example, as illustrated in FIG. 3 , a check is made as to whether ashadow associated with the target object (airborne object 101) is castbelow the target object (on the surface of the road 100) in the image10A. When the shadow is observed, the target object can be estimated asthe airborne object 101.

In Step S12, the drive assist apparatus 1 acquires feature informationof the recognized airborne object 101. As described above, the featureinformation of the airborne object 101 is, for example, information onthe size, falling speed, size change amount, and transparency of theairborne object 101. For example, the feature information of theairborne object 101 can be acquired from a result of the predeterminedimage processing performed based on the image data acquired by thecamera unit 10.

The mass and the falling speed of an object have a relationship of“falling speed of object=(object mass×gravitational acceleration)/airresistance”. According to this relationship, the falling speed of theobject decreases as the mass of the object decreases. In other words,determination can be made that the mass of the object decreases as thefalling speed decreases.

In Step S13, the drive assist apparatus 1 checks whether the size of therecognized airborne object 101 (area of an object image in the image10A) is smaller than a predetermined threshold. When the size of theairborne object 101 (area in the image) is smaller than thepredetermined threshold, the process proceeds to Step S14. Otherwise,the process proceeds to Step S18.

In Step S14, the drive assist apparatus 1 checks whether the fallingspeed of the recognized airborne object 101 is lower than apredetermined threshold. When the falling speed of the airborne object101 is lower than the predetermined threshold, the process proceeds toStep S15. Otherwise, the process proceeds to Step S18.

In Step S15, the drive assist apparatus 1 checks whether the size of therecognized airborne object 101 is changing. When the size is changing,the drive assist apparatus 1 checks whether the size change amount isequal to or larger than a predetermined threshold. When the size of theairborne object 101 is changing and the size change amount is equal toor larger than the threshold, the process proceeds to Step S16.

When the size of the airborne object 101 is not changing or when thesize is changing and the size change amount is smaller than thethreshold, the process proceeds to Step S18.

In Step S16, the drive assist apparatus 1 determines the recognizedairborne object 101 as an object that the vehicle M can pass even if theobject is not regarded as an avoidance target (hereinafter referred toas “passable object”).

In Step S17, the drive assist apparatus 1 checks whether thetransparency of the recognized airborne object 101 is equal to or higherthan a predetermined threshold (high transparency means “close tocolorless and transparent”).

This check step is provided for the following reason. In general,plastic bags have varying transparencies such as “colorless andtransparent”, “semi-transparent”, and “colored”. In view of this fact,the drive assist apparatus 1 of this embodiment checks the transparencyof the target object determined as the passable object through thechecks under the conditions described above (size, falling speed, andsize change) in the processes of Steps S13 to S15. The result showingthat the object has a low transparency (colored) serves as a criterionof whether to alert the driver for caution in a process of Step S20.That is, the drive assist apparatus 1 checks that the transparency ofthe target object is low in the process of Step S17, and adds the resultas the criterion of the determination as to whether to give an alert.

The transparency of the target object can be checked by the followingmethod. For example, the pixel value of the target object (airborneobject 101) and the pixel value of the surroundings may be comparedbased on image data acquired by the camera unit 10. Further, reflectionof light having passed through the target object (airborne object 101)may be detected and the degree of reflectance may be checked based onsurrounding environment data acquired by using the LiDAR sensor or thelike.

When the transparency of the airborne object 101 is equal to or higherthan the predetermined threshold in the process of Step S17, thereliability of the determination as the passable object increases. Inthis case, the normal traveling control is continued without taking anaction for the avoidance control. Then, the series of processes isfinished, and the process returns to the initial step (RETURN).

When the transparency of the airborne object 101 is lower than thepredetermined threshold in the process of Step S17, the process proceedsto Step S20.

In Step S20, the drive assist apparatus 1 alerts the driver that theairborne object 101 is present ahead. The alert is a visual alertdisplayed as an image on the HMI 31 (display panel), or an audio alertgiven by voice or sound using the HMI 31 (loudspeaker). In oneembodiment, the display panel and the loudspeaker of the HMI 31 mayserve as a notification device configured to notify the driver aboutpredetermined information. Then, the series of processes is finished,and the process returns to the initial step (RETURN).

The process of Step S20 (alert process) is performed for the followingreason. In the process of Step S20, the alert is given about theairborne object 101 identified as the passable object in the processesof Steps S13 to S16 and determined to have a low transparency (lowerthan the predetermined threshold) in the process of Step S17. Thisairborne object 101 is the passable object unlikely to hinder thetraveling of the vehicle M but may, for example, obstruct the forwardview of the vehicle M due to the low transparency. Therefore, when theobject is determined as the airborne object 101, no action is taken forthe avoidance control but the driver is cautioned.

When the process proceeds to Step S18 from Step S13, S14, or S15, thedrive assist apparatus 1 determines in Step S18 that the recognizedairborne object 101 is an object that may hinder the traveling of thevehicle M (hereinafter referred to as “impassable object”).

In Step S19, the drive assist apparatus 1 executes predeterminedemergency braking control. Then, the series of processes is finished,and the process returns to the initial step (RETURN). In this case, theemergency braking control is substantially the same as the control to beperformed in response to recognition of general obstacles.

According to the first embodiment described above, when the vehicle Mincluding the drive assist apparatus 1 is traveling along a road whilerecognizing the surrounding environment ahead in the traveling directionby using the surrounding environment recognition device (10, 37), thefeature information of the target object is acquired in the recognizedsurrounding environment. The traveling control unit 14 determineswhether the target object may hinder the traveling of the vehicle Mbased on the acquired feature information. When determination is madethat the target object has no possibility of hindering the traveling ofthe vehicle M, the normal traveling control for the vehicle M iscontinued.

With this configuration, when the target object among the recognizedthree-dimensional objects is the airborne object 101 having nopossibility of hindering the traveling of the vehicle M, the obstacleavoidance control can be suppressed. Thus, it is possible to constantlyexecute smooth drive assist control without causing driver's discomfort.

In this embodiment, the transparency is checked for the objectdetermined as the airborne object 101. When the transparency of theairborne object 101 is lower than the predetermined threshold, therecognized target object is the airborne object 101 but is nottransparent (but colored), thereby having a possibility of obstructingthe forward view of the vehicle M. In this case, the obstacle avoidancecontrol is suppressed and the driver is cautioned. Thus, the driveassist apparatus 1 of this embodiment can achieve more reliable andsmooth traveling control without causing driver's discomfort.

A drive assist apparatus 1 according to a second embodiment of thedisclosure is described. The drive assist apparatus 1 of this embodimentbasically has a configuration similar to that in the first embodiment,and slightly differs in terms of the processing flow. In the followingdescription, illustration and description are omitted for theconfiguration of the drive assist apparatus and the situation involvingthe drive assist apparatus, but FIG. 1 to FIG. 3 are referenced as inthe first embodiment. Operations of the drive assist apparatus of thisembodiment are described with reference to FIG. 5 .

The processing flow of FIG. 5 is executed in the same situation as thatin the first embodiment. The processes of Steps S11 and S12 in theprocessing flow of this embodiment (see FIG. 5 ) are the same as thosein the processing flow of the first embodiment (see FIG. 4 ). That is,the drive assist apparatus 1 recognizes an airborne object 101 in StepS11 of FIG. 5 . In Step S12 of FIG. 5 , the drive assist apparatus 1acquires feature information of the recognized airborne object 101.

In Step S21 of FIG. 5 , the drive assist apparatus 1 calculatesevaluation values for the individual conditions (size, falling speed,size change, and transparency of the airborne object 101) based on thefeature information of the airborne object 101 acquired in the processof Step S12.

The evaluation values of the individual conditions are defined asfollows. For example, the evaluation value of the size of the targetobject is defined on a scale of 0: “small”, 1: “medium”, and 2: “large”.The evaluation value of the falling speed of the target object isdefined on a scale of 0: “low”, 1: “medium”, and 2: “high”. Theevaluation value of the size change amount of the target object isdefined on a scale of 0: “small”, 1: “medium”, and 2: “large”. Theevaluation value of each condition is defined in the three levels, butis not limited to this example.

In Step S22, the drive assist apparatus 1 checks whether the total ofthe evaluation values calculated in the process of Step S21 is smallerthan a first threshold. For example, the first threshold is “2”. Thefirst threshold is not limited to “2”.

When the total evaluation value is smaller than the first threshold of“2”, the process proceeds to Step S23. When the total evaluation valueis equal to or larger than the first threshold of “2”, the processproceeds to Step S24.

In Step S23, the drive assist apparatus 1 determines the recognizedairborne object 101 as a passable object. This processing step is thesame as Step S16 of FIG. 4 .

In Step S24, the drive assist apparatus 1 checks whether the total ofthe evaluation values calculated in the process of Step S21 is equal toor larger than a second threshold. For example, the second threshold is“5”. The second threshold is not limited to “5”.

When the total evaluation value is equal to or larger than the secondthreshold of “5”, the process proceeds to Step S25. When the totalevaluation value is smaller than the second threshold of “5”, theprocess proceeds to Step S27.

That is, when the process proceeds to Step S27 from Step S24, the totalevaluation value is equal to or larger than the first threshold of “2”and smaller than the second threshold of “5” (the total evaluation valueis 2 to 4).

In Step S27, the drive assist apparatus 1 checks whether thetransparency of the recognized airborne object 101 is equal to or higherthan the predetermined threshold. When the transparency of the airborneobject 101 is equal to or higher than the predetermined threshold, thenormal traveling control is continued without taking an action for theavoidance control. Then, the series of processes is finished, and theprocess returns to the initial step (RETURN). When the transparency ofthe airborne object 101 is lower than the predetermined threshold, theprocess proceeds to Step S28. The process of Step S27 is the same asStep S17 of FIG. 4 .

In Step S28, the drive assist apparatus 1 alerts the driver that theairborne object 101 is present ahead. Then, the series of processes isfinished, and the process returns to the initial step (RETURN). Theprocess of Step S28 is the same as Step S20 of FIG. 4 .

When the total evaluation value is equal to or larger than the secondthreshold in the process of Step S24, the process proceeds to Step S25.In Step S25, the drive assist apparatus 1 determines the recognizedairborne object 101 as an impassable object. The process of Step S25 isthe same as Step S18 of FIG. 4 .

In Step S26, the drive assist apparatus 1 executes predeterminedemergency braking control. Then, the series of processes is finished,and the process returns to the initial step (RETURN). The process ofStep S26 is the same as Step S19 of FIG. 4 .

According to the second embodiment described above, the same effects asthose of the first embodiment can be attained. According to thisembodiment, the evaluation values are calculated for the individualconditions (size, falling speed, size change, and transparency of theairborne object 101) based on the feature information of the airborneobject 101, and determination is made as to whether the object is thepassable object based on the total evaluation value. Thus, it ispossible to make more reliable determination.

The embodiment of the disclosure is not limited to the embodimentsdescribed above, and various modifications and applications may be madewithout departing from the gist of the disclosure. The embodimentsinclude various aspects of the disclosure that may be extracted by anyappropriate combination of the disclosed constituent elements. Forexample, some of the constituent elements in the embodiments may beomitted as long as the problems described above can be solved and theeffects described above can be attained. The constituent elements ofdifferent embodiments may be combined as appropriate. The embodiment ofthe disclosure is limited to the appended claims but not limited tospecific modes of implementation.

According to the embodiments of the disclosure, it is possible toprovide the vehicle drive assist apparatus configured to perform thedrive assist control for avoiding collision with obstacles etc. based onthe surrounding environment information acquired by using the on-boardcamera device and the like, and configured not to immediately executethe obstacle avoidance control including the emergency braking controlwhen the airborne object or the like is recognized during the travelingof the vehicle, thereby constantly executing smooth traveling controlwithout causing driver's discomfort.

The traveling ECU 14 illustrated in FIG. 1 can be implemented bycircuitry including at least one semiconductor integrated circuit suchas at least one processor (e.g., a central processing unit (CPU)), atleast one application specific integrated circuit (ASIC), and/or atleast one field programmable gate array (FPGA). At least one processorcan be configured, by reading instructions from at least one machinereadable tangible medium, to perform all or a part of functions of thetraveling ECU 14. Such a medium may take many forms, including, but notlimited to, any type of magnetic medium such as a hard disk, any type ofoptical medium such as a CD and a DVD, any type of semiconductor memory(i.e., semiconductor circuit) such as a volatile memory and anon-volatile memory. The volatile memory may include a DRAM and a SRAM,and the non-volatile memory may include a ROM and a NVRAM. The ASIC isan integrated circuit (IC) customized to perform, and the FPGA is anintegrated circuit designed to be configured after manufacturing inorder to perform, all or a part of the functions of the modulesillustrated in FIG. 1 .

1. A vehicle drive assist apparatus to be applied to a vehicle, thevehicle drive assist apparatus being configured to execute at leastemergency braking control for avoiding collision of the vehicle with arecognized object, the vehicle drive assist apparatus comprising: asurrounding environment recognition device comprising: a recognizerconfigured to recognize a surrounding environment around the vehicle;and a feature information acquirer configured to acquire featureinformation of a target object in the recognized surroundingenvironment; and a traveling control unit configured to centrallycontrol a whole of the vehicle, wherein the traveling control unitcomprises a determiner configured to determine, based on the acquiredfeature information, whether the target object has a possibility ofhindering traveling of the vehicle, and the traveling control unit beingconfigured to continue normal traveling control for the vehicle when thetarget object is determined to have no possibility of hindering thetraveling of the vehicle.
 2. The vehicle drive assist apparatusaccording to claim 1, wherein the surrounding environment recognitiondevice is a stereo camera.
 3. The vehicle drive assist apparatusaccording to claim 1, wherein the feature information of the targetobject comprises: information related to a size of the target object;information related to a falling speed of the target object; andinformation related to a size change amount of the target object.
 4. Thevehicle drive assist apparatus according to claim 2, wherein the featureinformation of the target object comprises: information related to asize of the target object; information related to a falling speed of thetarget object; and information related to a size change amount of thetarget object.
 5. The vehicle drive assist apparatus according to claim1, further comprising a notification device configured to notify adriver who drives the vehicle about predetermined information, whereinthe feature information acquirer is configured to acquire informationrelated to a transparency of the target object as the featureinformation of the target object, and wherein the notification device isconfigured to notify the driver about predetermined alert informationwhen the transparency of the target object is determined to be lowerthan a predetermined threshold based on the acquired information relatedto the transparency.
 6. The vehicle drive assist apparatus according toclaim 2, further comprising a notification device configured to notify adriver who drives the vehicle about predetermined information, whereinthe feature information acquirer is configured to acquire informationrelated to a transparency of the target object as the featureinformation of the target object, and wherein the notification device isconfigured to notify the driver about predetermined alert informationwhen the transparency of the target object is determined to be lowerthan a predetermined threshold based on the acquired information relatedto the transparency.
 7. The vehicle drive assist apparatus according toclaim 3, further comprising a notification device configured to notify adriver who drives the vehicle about predetermined information, whereinthe feature information acquirer is configured to acquire informationrelated to a transparency of the target object as the featureinformation of the target object, and wherein the notification device isconfigured to notify the driver about predetermined alert informationwhen the transparency of the target object is determined to be lowerthan a predetermined threshold based on the acquired information relatedto the transparency.
 8. The vehicle drive assist apparatus according toclaim 4, further comprising a notification device configured to notify adriver who drives the vehicle about predetermined information, whereinthe feature information acquirer is configured to acquire informationrelated to a transparency of the target object as the featureinformation of the target object, and wherein the notification device isconfigured to notify the driver about predetermined alert informationwhen the transparency of the target object is determined to be lowerthan a predetermined threshold based on the acquired information relatedto the transparency.
 9. The vehicle drive assist apparatus according toclaim 5, wherein the notification device comprises one or both of adisplay panel and a loudspeaker.
 10. The vehicle drive assist apparatusaccording to claim 6, wherein the notification device comprises one orboth of a display panel and a loudspeaker.
 11. The vehicle drive assistapparatus according to claim 7, wherein the notification devicecomprises one or both of a display panel and a loudspeaker.
 12. Thevehicle drive assist apparatus according to claim 8, wherein thenotification device comprises one or both of a display panel and aloudspeaker.
 13. A vehicle drive assist apparatus to be applied to avehicle, the vehicle drive assist apparatus being configured to executeat least emergency braking control for avoiding collision of the vehiclewith a recognized object, the vehicle drive assist apparatus comprising:a surrounding environment recognition device comprising a sensor andconfigured to recognize a surrounding environment around the vehicle,and acquire feature information of a target object in the recognizedsurrounding environment; and circuitry configured to centrally control awhole of the vehicle, determine, based on the acquired featureinformation whether the target object has a possibility of hinderingtraveling of the vehicle, and upon determining that the target object ishas no possibility of hindering the traveling of the vehicle, continuenormal traveling control for the vehicle.